The amount of attention paid to Kubernetes has increased substantially over the past couple of years. What started out as a relatively obscure container management system open sourced by Google has turned into the must-have technology for running machine learning and advanced analytics applications, among other workloads. But is Kubernetes the real deal? Will K8s deliver on the hype, or turn into just another once-shiny thing that lost its luster? Kubernetes certainly seems to be the right technology for the right time.

Red Hat, Inc., the world's leading provider of open source solutions, today highlighted that more organizations are using Red Hat OpenShift as the foundation for building artificial intelligence (AI) and machine-learning (ML) data science workflows and AI-powered intelligent applications. OpenShift helps to provide agility, flexibility, portability and scalability across the hybrid cloud, from cloud infrastructure to edge computing deployments, a necessity for developing and deploying ML models and intelligent applications into production more quickly and without vendor lock-in. AI/ML represents a top emerging workload for Red Hat OpenShift across hybrid cloud and multicloud deployments for both our customers and for our partners supporting these global organizations. By applying DevOps to AI/ML on the industry's most comprehensive enterprise Kubernetes platform, IT organizations want to pair the agility and flexibility of industry best practices with the promise and power of intelligent workloads. As a production-proven enterprise container and Kubernetes platform, OpenShift delivers integrated DevOps capabilities for independent software vendors (ISVs) via Kubernetes Operators and NVIDIA GPU-powered infrastructure platforms.

Artificial intelligence and machine learning are critical tools that data scientists have to master.

Apache Airflow is a popular platform to create, schedule and monitor workflows in Python. It has more than 15k stars on Github and it's used by data engineers at companies of all sizes including Twitter, Airbnb and Spotify. If you're using Apache Airflow, your architecture has probably evolved based on the number of tasks and their requirements. While working at Skillup, we first had a few hundred DAGs to execute all our data engineering tasks. Then we started doing machine learning.